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Abstract 



Educational researchers have employed various self-efficacy instruments in a wide spectrum of disciplines and 
academic settings. However, self-efficacy measures specific to the online environment have not been developed yet. 
This paper provides a brief history of the online environment and discusses the development and validation of an 
instrument that measures online students* self-efficacy beliefs with communication technologies such as email, 
Internet, and computer conferencing. Content validity, construct validity, and reliability were established in order to 
validate this instrument. Factor analysis and correlational analysis revealed that all items could be collapsed into 
one scale. This indicated that there is only one unified construct for online self-efficacy. The Cronbach's Coefficient 
Alpha for the whole instrument was 0.95. 
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Validation of the Online Technologies Self-efficacy Scale (OTSES) 



Marios Miltiadou and Chong Ho Yu, Ph.D. 
Arizona State University 



Introduction 

Online education is one of the most dynamic and enriching forms of learning that exist today. The online 
environment offers appealing educational alternatives and provides life-long learning opportunities for those whom 
a traditional university setting does not work. Online education depends on the Internet and computer-mediated 
communication (CMC) systems for the delivery of instruction and interaction between students and instructors. 

Computer-mediated communication is still a fairly new development in education and many online students 
encounter various difficulties with such technologies. Novice students, for example, tend to feel apprehensive about 
using CMC systems and the Internet in ways that may jeopardize intellectual interaction and their ability to succeed 
in an online course. Students who do not feel comfortable with online technologies tend to spend more time trying to 
figure out how to use them in order to communicate with instructors, submit online assignments, or download class- 
related material from the course's web site. As a result, these students tend to spend less time working on the actual 
course content. Additional research is needed in order to determine students’ self-efficacy beliefs with online 
technologies. Such findings would enable instructors to provide immediate remediation to students early in the 
semester. Such actions might increase interaction and lower attrition rates. 

The purpose of this study was to develop and validate a new instrument that measured students’ confidence 
levels with online technologies. The following sections first introduce the concept of CMC and then continue with a 
description of both the theoretical concept and related research for self-efficacy. The next section describes the 
methodology for developing and validating the instrument. 

Computer-mediated Communication in the Online Classroom 

While technology in general is the backbone of the virtual environment, CMC is the gateway for thousands 
of online learners in virtual communities. According to Harasim (1996) CMC is becoming the leading way to reach 
distance learners and proving to be a global communication system. CMC refers to the use of networked computers 
for communication, interaction, and exchange of information between students and instructors (Berge & Collins, 
1995). Examples of CMC technologies include electronic mail, bulletin boards, newsgroups, and computer 
conferencing. 

Computer-mediated communication is characterized by a highly interactive, multi-way synchronous or 
asynchronous communication (Romiszowski & Mason, 1996). Synchronous interaction allows students and 
instructors to exchange ideas and discuss course topics at the same time via a virtual discussion area. Asynchronous 
interaction provides opportunities for active input from all members of the online classroom and supports learner- 
centered learning environments. For example, CMC allows for one-to-many or many-to-many interaction, which 
encourages conversation and collaboration between peers as well as engagement on task and sharing of information 
and ideas (Jonassen, Davidson, Collins, Campbell, & Bannan Haag, 1995). 

The rapid growth of computer networks and the evolution of the Internet in the last decade have magnified 
the use of CMC to the point that it plays an essential role in the online delivery of instruction. Riel (1993) stated that 
online learners interact with their peers, instructors, and content experts in ways that allow students to develop their 
critical and problem solving skills. In the same context, Harasim (1990a) stated that CMC enables online students to 
participate in active learning. Furthermore, research studies found that the interaction of students and instructors via 
CMC positively affected student outcomes and contributed to their learning (Harasim, 1990b; Miller & Webster, 
1997; Waggoner, 1992). 
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Self-efficacy 



Self-efficacy is a major component of Bandura’s (1986) social cognitive learning theory. Bandura described 
self-efficacy as individuals’ confidence in their ability to control their thoughts, feelings, and actions, and therefore 
influence an outcome. These perceptions of self-efficacy influence individuals' (a) actual performance (Locke, 
Frederick, Lee, & Bobko, 1984; Schunk, 1981), (b) emotions (Bandura, Adams, & Beyer, 1977; Stumpf, Brief, & 
Hartman, 1987), (c) choices of behavior (Betz & Hackett, 1981), and (d) amount of effort and perseverance 
expended on an activity (Brown & Inouye, 1978). 

According to Bandura (1986), individuals acquire information to help them assess self-efficacy from four 
principal sources: (a) actual experiences, (b) vicarious experiences, (c) verbal persuasion, and (d) physiological 
indexes. Individuals’ own performances, especially past successes and failures, offer the most reliable source for 
assessing efficacy. Observation of similar peers performing a task conveys to observers that they too are capable of 
accomplishing that task. A form of verbal persuasion is when individuals are encouraged to believe that they possess 
the capabilities to perform a task (e.g. being told "you can do this’’). Finally, individuals might interpret bodily 
symptoms such as increased heart rate or sweating as a signal for anxiety or fear, resulting in an indication of their 
own lack of skills. 

Various researchers have established that self-efficacy is a strong predictor of academic performance and 
course satisfaction in traditional face-to-face classrooms. Multon, Brown, and Lent (1991) reviewed a 
comprehensive list of studies that examined self-efficacy in achievement situations. Findings suggested that self- 
efficacy beliefs were positively related to academic performance. In the same context, Ames (1984) and Nicholls 
and Miller (1994) suggested that students’ self-perceptions of ability are positively related to achievement and 
student motivation. 



Theoretical Basis for Developing the New Instrument 

According to Bandura (1986), individuals make personal ability judgements and evaluations through a 
cognitive appraisal system that is specific to the individual, the task, and the particular situation at any given 
moment. Bandura (1986) cautioned that a self-efficacy instrument must assess the specific skills needed for 
performing an activity and must be administered during the time that the performance is being assessed. Vispoel & 
Chen (1990) stated that no single standardized measure of self-efficacy is appropriate for all studies and advised 
researchers to develop new or significantly revise existing measures for each study. 

A review of the literature revealed no instruments specific to measuring online students' perceptions of self- 
efficacy with online technologies. A few articles were identified where authors developed and validated instruments 
specific to general computer technologies (Burkhardt & Brass, 1990; Compeau & Higgins, 1995; Delcourt & 
Kinzie, 1993; Hill, Smith, & Mann, 1987; Murphy, Coover, & Owen, 1989). Such computer technologies consisted 
of general computer skills such as file management. In addition, the instruments measured students' efficacy beliefs 
with software applications such as word processing, spreadsheets, databases, or statistical programs. One instrument 
included a subscale about electronic mail (Delcourt & Kinzie, 1993). 

In light of the importance of self-efficacy in predicting academic achievement, and the absence of specific 
instruments in the context of the online environment, the authors of this paper developed a new instrument for 
measuring students' self-efficacy beliefs with online technologies. 

Methodology 

The instrument development process was based on the recommendations of Crocker and Algina (1986). A 
pool of 40 items was fnst constructed and for each item a set of matching objectives was created identifying 
behaviors to represent each construct. Feedback received from content experts, students, and survey designers from 
various educational institutions enabled the researchers to improve the instrument. Ten items were deleted from the 
original pool for they offered no new information to the construct. The final instrument consisted of 30, 4-point 
Likert-scaled items. For each item, students were asked to indicate their level of confidence from "Very Confident," 
"Somewhat Confident," "Not Very Confident," to "Not Confident At All." Each statement was preceded by the 
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phrase "I feel confident...” Students were asked to select the option ”Not Confident At All" if they did not know 
what the statement meant. 

The researchers identified four subscales: (a) Internet Competencies, (b) Synchronous Interaction, (c) 
Asynchronous Interaction I, and (d) Asynchronous Interaction II. The Internet Competencies subscale (Table 1) 
contained 10 items about the use of an application (such as Netscape or Explorer) that enabled participants to use the 
Internet. 



I would feel confident... 

1. Opening a web browser (e. g. Netscape or Explorer) 

2. Reading text from a web site 

3. Clicking on a link to visit a specific web site 

4. Accessing a specific web site by typing the address (URL) 

5. Bookmarking a web site 

6. Printing a web site 

7. Conducting an Internet search using one or more keywords 

8. Downloading (saving) an image from a web site to a disk 

9. Copying a block of text from a web site and pasting it to a document in a word processor 

10. Creating a simple web page with text, images, and links 
Table 1. Internet Competencies Subscale 

The Synchronous Interaction subscale (Table 2) contained four items about the use of a synchronous chat 
system (such as Courseinfo, First Class, NetMeeting, or IRC) that enabled participants who were online at the same 
time to communicate with each other. 



I would feel confident... 

11. Providing a nickname within a s)mchronous chat system (if necessary) 

12. Reading messages from one or more members of the synchronous chat system 

13. Answering a message or providing my own message in a synchronous chat system (one-to-many interaction) 

14. Interacting privately with one member of the synchronous chat system (one-to-one interaction) 

Table 2. Synchronous Interaction Subscale 

The Asynchronous Interaction I subscale (Table 3) contained nine items about the use of an electronic mail 
system (such as Pine, Netscape Mail, or Outlook) that enabled participants who were not online at the same time to 
communicate with other people. 



I would feel confident... 

15. Logging on and off an e-mail system 

16. Sending an e-mail message to a specific person (one-to-one interaction) 

17. Sending one e-mail message to more than one person at the same time (one-to- many interaction) 

18. Replying to an e-mail message 

19. Forwarding an e-mail message 

20. Deleting messages received via e-mail 

2 1 . Creating an address book 

22. Saving a file attached to an e-mail message to a local disk and then viewing the contents of that file 

23. Attaching a file (image or text) to an e-mail message and then sending it off 

Table 3. Asynchronous Interaction I Subscale 

The Asynchronous Interaction II subscale (Table 4) contained seven items about the use of a newsgroup, a 
bulletin board, or the discussion board of a conferencing system (such as Courseinfo or FirstClass) that enabled 
participants who were not online at the same time to post messages or reply to messages. 



I would feel confident... 

24. Signing on and off an asynchronous conferencing system 

25. Posting a new message to an synchronous conferencing system (creating a new thread) 
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26. Reading a message posted on an asynchronous conferencing system 

27. Replying to a message posted on an asynchronous conferencing system so that all members can view it 

28. Replying to a message posted on an asynchronous conferencing system so that only one member can view it 

29. Downloading (saving) a file from an asynchronous conferencing system to a local disk 

30. Uploading (sending) a file to an asynchronous conferencing system 
Table 4. Asynchronous Interaction II Subscale 

The instrument was pilot-tested with 30 graduate students enrolled in various online graduate courses at a 
major southwestern university. Minor revisions pertaining to the language and grammar of the instrument were 
performed. 

Participants, Setting, and Data Collection 

Approximately 330 college level students enrolled in several online courses at five southwestern 

educational institutions participated in the study during the first week of the spring 2000 semester. All participants 

did not have any formal instruction on using online technologies. Some instructors offered a voluntary technology 
orientation meeting, where for two hours students gathered on campus and instructors explained the use of online 
technologies and course requirements. In such cases all students filled out a paper and pencil instrument prior to any 
instruction. The rest of the students who did not attend the technology orientation sessions filled out an online 
version of the instrument before their first assignment was due. Students* responses were collected via email. 

Statistical Analysis 

Construct validity and internal consistency of the instrument were assessed in order to validate the survey. 
Construct validity is a measure of how meaningful the instrument is in practical use, in which an inference can be 
drawn form test scores to a psychological construct. Latent constructs were triangulated by different manifest 
indicators and were computed by factor analysis. Prior to conducting factor analysis, the instrument was comprised 
of four subscales. After running factor analysis, it was found that items could not be distinctly loaded into four 
subscales. Correlational analysis also revealed that the four subscales were highly inter-related. It was concluded 
that all subscales could be collapsed into a single construct. Furthermore, item 10 (creating a simple web page with 
text, images, and links) was deleted because the factor loading was indetermined. An internal consistency reliability 
(Cronbach’s coefficient alpha) estimate of .95 was obtained for the entire 29-item instrument. The final instrument 
can be found in the Appendix. 

Conclusion 

The purpose of this study was to develop and validate an instrument that measured online students’ self- 
efficacy beliefs with communication technologies such as email, Internet, and computer conferencing. Content 
validity, construct validity, and reliability of the instrument were established in order to validate the survey. Factor 
analysis and correlational analysis revealed that items in the instrument could be collapsed into one scale. The 
Cronbach’s Coefficient Alpha for the whole instrument was 0.95. 

The development of the Online Technologies Self-efficacy Scale could benefit both instructors and students 
involved with online courses. By using this instrument, instructors could identify students who do not feel confident 
with online technologies at the beginning of an online course. Appropriate actions would then be taken so that 
students' efficacy perceptions with online technologies would increase. For example, instructors could show students 
how to use online technologies, or advise them to practice their computer skills using a tutorial. Furthermore, 
instructors could pair up students in order to help each other, and provide effective and positive feedback in order to 
increase their motivation. The provision of early feedback and remediation could result in students persisting in the 
course. This may translate to a decrease in the high attrition rates evidenced in online courses. 
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APPENDIX 



ONLINE TECHNOLOGIES SELF-EFFICACY SCALE (OTSES) 

Thank you for agreeing to fill out this questionnaire. The following questions ask how confident you feel 
with using online technologies (such as Internet, email, etc.) in order to succeed in an online course. 

If you do not have much computer experience, just complete the questionnaire to the best of your 
knowledge. DO NOT WORRY! Remember that each section begins with the statement "I would feel confident..." 
performing an activity, and not "I have done it before." It does not matter whether you have had experience with the 
activities described. We would like to find out what your perceptions are performing the activities below. There are 
no right or wrong answers, just answer as accurately as possible. 

Please read the directions below and then fill in ALL items 



The survey requires you to indicate your level of confidence with the statements below by writing an 8or an 
4in each box from "Very Confident" to "Not Confident At All". If you do not know what a statement means, choose 
"Not Confident At All." 



A) Questions about using the Internet (Internet Competencies) 


I would feel confident... 


Very 

Confident 


Somewhat 

Confident 


Not Very 
Confident 


Not 

Confident 
At All 


1. Opening a web browser (e. g. Netscape or Explorer) 










2. Reading text from a web site 










3. Clicking on a link to visit a specific web site 










4. Accessing a specific web site by typing the address (URL) 










5. Bookmarking a web site 










6. Printing a web site 










7. Conducting an Internet search using one or more keywords 










8. Downloading (saving) an image from a web site to a disk 










9. Copying a block of text from a web site and pasting it to a 
document in a word processor 
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(B) Questions about chatting "live" via a synchronous chat system such as Courseinfo, First Class, NetMeeting, or 
IRC (some people call it Synchronous Interaction) 


I would feel confident... 


Very 

Confident 


Somewhat 

Confident 


Not Very 
Confident 


Not 

Confident 
At All 


10. Providing a nickname within a synchronous chat system 
(if necessary) 










1 1 . Reading messages from one or more members of the 
synchronous chat system 










12. Answering a message or providing my own message in a 
synchronous chat system (one-to-many interaction) 










13. Interacting privately with one member of the 
synchronous chat system (one-to-one interaction) 











(C) Questions about using an e-mail system such as Pine, Netscape Mail, or Outlook to communicate with friends, 
instructors, or other students who are not online at the same time (Asynchronous interaction I) 


I would feel confident... 


Very 

Confident 


Somewhat 

Confident 


Not Very 
Confident 


Not 

Confident 
At All 


14. Logging on and off an e-mail system 










15. Sending an e-mail message to a specific person (one-to- 
one interaction) 










16. Sending one e-mail message to more than one person at 
the same time (one-to- many interaction) 










17. Replying to an e-mail message 










18. Forwarding an e-mail message 










19. Deleting messages received via e-mail 










20. Creating an address book 










21. Saving a file attached to an e-mail message to a local disk 
and then viewing the contents of that file 










22. Attaching a file (image or text) to an e-mail message and 
then sending it off 











(D) Questions about posting a message to a newsgroup, a bulletin board, or on the discussion board of a 
conferencing system (such as Courseinfo, FirstClass, etc.) where participants are not online at the same time 
(Asynchronous interaction II) 


I would feel confident... 


Very 

Confident 


Somewhat 

Confident 


Not Very 
Confident 


Not 

Confident 
At All 


23. Signing on and off an asynchronous conferencing system 










24. Posting a new message to an synchronous conferencing 
system (creating a new thread) 










25. Reading a message posted on an asynchronous 
conferencing system 










26. Replying to a message posted on an asynchronous 
conferencing system so that all members can view it 










27. Replying to a message posted on an asynchronous 

conferencing system so that only one member can view 
it (reply to sender) 










28. Downloading (saving) a file from an asynchronous 
conferencing system to a local disk 










29. Uploading (sending) a file to an asynchronous 
conferencing system 
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